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ABSTRACT ' / 

The Banner in which instructions to respondents 
influenced the psychoBetric results of an affective instruBent was 
exaBined. undergraduate students enrolled in an educational 
neasursBent course were randoBly assigned to either iBBediate 
response or careful response treataents. A f if teen-itea attitude 
scale «as adainistered to both groups. The iaaediate reaction group 
was instructed to rely on their first iapressions of the iteas and 
were advised not to go back once they had responded to a stateaent. 
The careful response group was instructed to be as sincere and 
accarate as possible in their answers and to feel free to change 
responses if necessary. For each group, coefficient alpha was 
coaputed as an estiaate of internal consistency. The t-test and 
discriainant analysis were used to contrast the two data sets. Mo 
itea types could be identified to differentiate between the two 
groups of respondents. In addition, instruaent reliability and total 
scores were not significantly different und^r either condition of 
instructions. (Author/JAC)^ 
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Abstract 

This study soiight- to determine hon' instructions to 
respondents influenced the psychometric results of an affec- 
tive instrument. Some respondents were asked to "immediately 
respond" to each item, while others were asjced to "^carefully 
respond". The results indicated that neither reliabilities 
nor total scores were adversly affected to any ma;for degree 
by either set of instructions, but it was suggested that 
perhaps the "carefully respond" type of instruction was more 
ethically defensible. 
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Effects of Type of Instructions 
on the Characteristics 
of an Affective Instrument 

Objectives of the Inquiry 
A cursory review of commonly used aff eotlTe measures 
(e.g. aoblnson. and Shaver, 1973; Bhaw and Wright, 1967) reveals 
a diversity in types of instructions, particularly with respect 

to the amount of reflection requested of the respondent prioir 

/ - ■ . ■ 

to /recording his overt response. Some instructions request 
l^edlate responses, others enc6urage careful reflection, while 
/yet others omit any reference' to the matter. 

Researchers in th« are^fi?:?* cognitive test construction , 
have investigated ft somewhat: related phenomenon* To answer 
the question, "Should ekaminees feel free to change their 
origlniil responses to objective test items?," most researchers 
have f ©Closed on the effects on total scores/ prith" reasohsbly 
considllfeht' findinge^ that response changes improve scores! more 
often than not. After reviewing the research literature of , 
the past fifty years, Mueller and* Wasser (1977) concluded 
that "from tkese studies it seems clear that 'students^ taking 
objective tests should be encouraged to deliberate on itepd » 
for which they rare not sure of the correct- answer" (p. 12). . 



EfJects'of Type of Instruct Ions 
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Recently, when Cropker and Benson, (1977) investigated ' 
the effects of examinee response changes upon oth'lar 'psychometri 
properties of cognitive tests, they fxjund tliat "test i^'ia- 
bility dees not appear to be adversly affected when examlneeB 
change their answers" (p. 4),' , * 

If respondents- to cognitive measures should be encouraged 
to deliberate on Items, how should respondentfi to affective / 
measures be instructed? The purpose of this investigation 
was to compare the effects of two extremes 6f instructions" ' 
on the psychometric properties of an affective- instrument. • 
In particular, the following questions were addressed: 

1. Is "^test reliability (internal consistency) affected? 

2. Is the total score affected? 

* 3* Are thiere particular types of items that are affected? 

"■ ■ ■ ^ ^• ^ , ~ ^ ' 

Sample and Instrument. ' ' 

/ \Bach of the eleven sections of an undergraduate educa- 
tional ^asurement course was randomly assigned to -one of two / 
treatments, designated" as either "Immediate Response" or "Care- 
•ful Response." The resulting two groups Contained 199 and 
226 subjects, respect iyelry. 

Selected as this 'instrument was a w^ll-researched ' attitude 
scale (Keri-^inger, *1067; p. 62, 247), ehtitled Education Scale 
Vll,' Subs c ale B (Traditionalism^. ' It /was chosen because of 

its appropriate iaubject matter^ (education), brevity (15 items), 

^ . V .' '"':. }. ' •• ^ ;■■ / , , ^ , - 

and reasonable reliability Xe.g.^re pair ted alpha values betwieen 
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- " : / ' " ^ • ■ ., ' .. •■ ■ \ . > 

V69 itnd •78)* The Instrument was modified frea ita or4.ginal ' 

■ ■ J ' ~ : ' . - •,■ ' , , -■ 

- 7'P^int ecale to a 5 .point Likert-type scale (strongly agree 
to /strongly 'iiisagteej. An illustrative "item ffi|llow8:.;i "The 

curriculum cohslats of subject-matter to be learned and skills', 

" / ' , ' ■ ■ , ' ' - ■ '■ 

to-be acquired," 

■'■/ . ' y ' ^ ' ■ , ■. ■ ■ ' '•■ 

Methods 

Each group W89 asked read, tii^ infltructione' and follow / 

• • '■' ■ ' . , ; ,^ ' #' / 

them jexactly. * 

"Immediately Respond" group: "Indicate yo'ur - immediate 

• « . . ■ * • 

reaction without thinking too much about it. Hemember that 

first tlapressions are usually best in such matters. Go rapidly; 

■ - ■ - ■, .. ,.1 • . 

do not go back once you *iiave marked a statement," 

J "Carefully Respond" group: ^Consider each question care-:, 

fully , ' Be as sincere and at^curate as Possible within the , 

limited time. If upon >a^fl^cti on, you; desire to alter "your 

response, please feel free to change it by erasing or marking 

it < out and circling another r<^sp6n8e." - * > 

To analyze^thev^sul^ing data, the five^esponse^ Strongly 

Disagree through Strongly Agree, were assigned the integers one ' 

through fife,,, respectively. For each experimental group, 

coefficient alpha (Cronbach, 1953f) was computed as an estimate 

of internal consistency;- The two data sets tv^re then contrasted 

with the-t-test on total scores and *with a discriminant 

analysis using Wilks Method XNie, et ai; .1975, P. ^7). 



•BffectB of Type «f Instructions • 

• - ■• . , _ . - ^ ■ . ■ ■ ■ 4 ■ 

- ' - • - ■ Result 8- ■ •• ■ 

The twol obtained Talues of coefficient' alpha (Table 1) 
were conpared using the test for Independent correlatien coef- 
fiClents (Hayri, 1973, -p. 663). a^he resulting i ralue of .^7 
did n«tVallow^ rajdctlon of the nCill h^^pothesls^ , even with a 



sfgnlfdlcance level as large ^8 IC^iin each tail. This larger 
than usual significance level was chosen to redupe the Type II 
error. When Owoa (1962» p. 518), in his. well-knowibo6k of 
mathematical tables, desired to show that a pal^ of samples *- 
were-Hessentlally equivalent , he compared them using a 10% ■ 
i^gniflcance. level in each tajLl. Accordingly, in this case, 
the "^use of tjkat larger significance level increases our con- . 
fidencfe that no impctrtant diffe5?enc6 exists between .the relia- 
bilities of the two forma of th^ attitude scale^ 

When the groups w'ere Gompared on total score, the result- 

•x'.' ^-L ■ ■■■■■■ ■ ■ ' 'S. 

ing difference of 1.3 points produced a t^value. of 2.p3» which 
was significant at the .05 levfel. : v 

To, see if particular items wete, differentially affected 
by the instructions, a discriminant analysis was performed. 
Wilks meJ;hod is a si^epwi^se solution, selecting at e.Ji^ch step 
that Tariable (item in this ca^e) ifhich maximizes the F ratie 
between group centroids. However, .this procedure did not 

yield any function signlfic'^t at the .05 level; thus^^ no * 

■■ * - / . ■ ■ • ■ ■ ■ : ■ . ■ I . . ;> 

particular item types could be identified tar'. differentiate 



between the twe groups o^ respondents. 
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"Educational Importance the Study 
On the basis of the obtained results, It appears that the 
instrument reliability was not . sig^nificantly different under 
either cendition -of instiiuctions. " The differences between the 
alpha Coefficients proved tp be noh-siSJiif^°a^* eren with av 
large rejection region. Further, it can be seen by inspection 
that these coefficients are in the vicinity. of those reported 
*by Kerlinge^Cl967), as was mentioned earlier in this paper. 
Although the t-test computed on the difference between" 
the mean scores of the groups proved to be statistically sig- 
nificant, it is not certain thdt'this should be' considered as 

• . s '. _ , 

hating practiQal^ importance. The differeiice between meaiis, in 
terms of, standard deviation units, is .designated as the "effect 
^size'' by Cohen (1969, p. 8). w For this comparison, the effect 
size proved to be onl3r".2, a value lajieled as "small" by Co'hen 
(1969^ p. 23). Purthel^mpre , the computation of omega sjjjia^re^l 
indicated- that the independent variable (type of instruct ions 
given) accounted for only about one percent of the va;riance in 
t He Scores (Hays, 1973» ;p. .-ilJ). There would probably be 
imiversal agreement that this,' indeed^ demonstrates a statis- 
tically significant yet trivial relationship between the t^pe . 
of instructions and the composite scores. , 

■ ' ' • . <» 

■ . ,' . , • 

' It might be hypothesized' tHat*^certain items were affected 
by -the instruction^; but that -they were not discernible by the 
t^-test because th6y wefe only a part of tl^ total score* But 
the nofi-signific ant results of the discriminant analysis' failed 
=to support this hypothesis. " , 
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The results of this study suggest that the • variations in 
directions di(jt not create substantial variation in the psycho- 
metric properties of the instrument or in/the level. of scores 
generated. The researcher and follower of research might be 
encouraged by these results , inasmuch a<s they can be interpreted, 
as indicating that questionnaire results are not likely to be 
seriously affected by the type of instructions given to the 

. ■ ' ' ' • » 

respondents. ^ 

However, the. generalizabilitj^. of these results may be 
.limited. In this study only undergraduate college^ students 
were used and only one test was administered. Perhaps the 
results would be different wi"th either more mature or less 
mature subjects, oir with different instruments.' Future studies 
might also examine item type, instruiment length, aubject , 
matter, etc. . ' 

Although the findings of this study did not show signifi- 
cant differences, perhaps. there is an overriding ethical 
consideifation of what is conveyed to the students responding 
to the instrument. Should pupils be encouraged to act upon 
'their . first impressions? One of the hallmarks of a civilized 
society is that the members carefully consider their thoughts ^ 
before translating them into action. Perhaps students can 
trbin themselves so that their first reactions are consistent 

I «. » a 

% . . ■ 

with careful reason, but it appears' doubtful that immedialie 

Judgments will be better than carefully reasoned ones. Encour- 

- .. ■ ' ■ • . ' ■ ' * . ■ ■ ' ' 

aging considered responses, as oppos-ed to advocating first 
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reactlona, isf desirable procedure even though it makes little 
di/fetence paychoffletrically, 
' A perusal of coaimonly used affective! Inatruments indicateB" 
that many authors hafia both written instructions requesting • 
careful Jiidgments and allocated .sufficient tj.me for this 
process (aa is usually:done with cognitive instruments )• 
P&rhaps this investi^ation/wili serve to further encourage 
that practice. 
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